User Verification by Combining Speech and Face Biometrics in Video

In this paper, physiological biometrics from face are combined with behavioral biometrics from speech in video to achieve robust user authentication. The choice of biometrics is motivated by user convenience and robustness to forgery as it is hard to simultaneously forge these two biometrics. We used the Mel Frequency Cepstral Coefficients for text-independent speaker recognition and local scale invariant features for video-based face recognition. Results of the two classifiers were fused using a weighted sum rule and an equal error rate of 0.6% was achieved on the VidTIMIT audio-visual database. We also performed identification experiments and achieved a combined identification rate of 99.13% on the same database.